Repository navigation
Load comments and users in chunks in wp comment list and wp user list - #666
Open
swissspidy wants to merge 3 commits into
Open
swissspidy wants to merge 3 commits into
swissspidy wants to merge 3 commits into
Conversation
…ist` Like `wp post list`, query the IDs of all matching comments or users first, then load the objects a chunk at a time and clear the object cache after each chunk. Combined with the formatter streaming CSV and JSON output for iterators, the memory use no longer grows with the number of items. If a displayed field is read through the object cache, like user meta or the post title of a comment, the cache is kept. Also only compute the `url` field of comments, users and terms when it is displayed. For comments it took about 40% of the time. The chunk size and the cache helper move to CommandWithDBObject. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D26yjkN2BiqCXT6p6o1WqS
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configuration
📒 Files selected for processing (5)
💤 Files with no reviewable changes (1)
Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review. 📝 WalkthroughWalkthroughComment and user listings now load eligible results in chunks and preserve query order. Comment, user, and term listings add URLs only when requested. Feature scenarios check large comment and user listings and CSV output. ChangesEntity listing updates
Priority: ➖ Normal Estimated code review effort: 3 (Moderate) | ~25 minutes Change: Refactor Sequence Diagram(s)sequenceDiagram
participant CLI
participant Comment_Command
participant Database
participant Formatter
participant RuntimeCache
CLI->>Comment_Command: request comment listing
Comment_Command->>Database: query matching comment IDs
Comment_Command->>Database: load comment batches
Comment_Command->>Formatter: yield comments in ID order
Comment_Command->>RuntimeCache: clear runtime cache after each batch
sequenceDiagram
participant CLI
participant User_Command
participant Database
participant Formatter
participant RuntimeCache
CLI->>User_Command: request user listing
User_Command->>Database: query matching user IDs
User_Command->>Database: load user batches
User_Command->>Formatter: yield users in ID order
User_Command->>RuntimeCache: clear runtime cache after each batch
Merge Risk: ⚪ Minimal · up to The change loads comments and users in chunks and computes URLs only when requested. No concrete merge-blocking risk is evident from the supplied context. The memory savings depend on companion changes in wp-cli core. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
command:comment-list
swissspidy
added this pull request to stack #667
October 6, 2026 16:45
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
wp-cli/wp-cli#6426 makes the formatter read the requested fields of each item while it is current, also when it can't stream them. User meta and the post fields of comments are therefore read before the cache of their chunk is cleared, so the cache no longer needs to be kept for them. The posts of each chunk of comments are now loaded in one query, so that fields like post_title don't load each post again in every chunk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D26yjkN2BiqCXT6p6o1WqS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #663. It applies the same approach as
wp post listto comments and users. Please merge #663 first; this PR's base will then switch tomain.Changes
wp comment listandwp user listquery the IDs of all matching items first, then load the objects a chunk at a time and clear the object cache after each chunk. Together with the formatter streaming iterators (Stream CSV and JSON output when items are given as an iterator wp-cli#6420 and follow-ups), memory no longer grows with the number of items.post_title) are read while each item is current (Only keep the requested fields of items from an iterator wp-cli#6426), so the cache can always be cleared. The posts of each chunk of comments are loaded in one query, so a post isn't loaded again for each of its comments.urlonly when displayed: theurlfield of comments, users and terms is now only computed when it's requested. For comments it was about 40% of the time.Post_CommandtoCommandWithDBObject.--format=idsand--format=countkeep their existing path.Results
--format=csvwp comment list, 100k commentswp user list, 50k usersRun time drops by about 40% for both. The output is identical to
main.With WP-CLI
main, on the same site:--format=csvwp user list --fields=ID,user_login,first_name,nicknamewp comment list --fields=comment_ID,post_titleTests
features/comment.featureandfeatures/user.feature. Each lists more items than one chunk (1,200 comments, 600 users) and checks:--format=ids,urlfield for comments, and a user meta field for users.comment.feature,user.feature,term.featureandpost.featurepass locally (92 scenarios) with WP-CLImain.🤖 Generated with Claude Code
https://claude.ai/code/session_01D26yjkN2BiqCXT6p6o1WqS
Summary by CodeRabbit